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\  ^ 

U-^Oinributed  problem  solving  network!  provide  an  interesting 
application  area  for  meta-level  control  through  the  use  of 
organizational  structuring.  We  describe  a  decentralized 
approach  to  network  coordination  that  relies  on  each  node 
making  sophisticated  local  decisions  that  balance  its  own 
perceptions  of  appropriate  problem  solving  activity  with 
activities  deemed  important  by  other  nodes.  Each  node  is 
guided  by  a  high-level  strategic  plan  for  cooperation  among 
the  nodes  in  the  network.  The  high-level  strategic  plan, 
which  is  a  form  of  meta-lcvel  control,  is  represented  as  a 
network  organizational  structure  that  specifies  in  a  general 
way  the  information  and  cootrol  relationships  among  the 
nodes.  An  implementation  of  these  ideas  is  briefly 
described  along  with  the  results  of  preliminary  experiments 
with  various  network  problem  solving  strategics  specified  via 
organizational  structuring.  In  addition  to  its  application  to 
Distributed  Artificial  Intelligence,  this  research  has 

implications  for  organizing  and  coatrolling  complex 
knowledge-based  systems  shat  involve  semi-autooomout 
problem  solving  agents.  , 

I  INTRODUCTION 

Cooperative  distributed  problem  solving  systems  are 
distributed  networks  of  semi-autonomous  processing  nodes 
that  work  together  to  solve  a  tinglt  problem.  Each  node  is 
a  sophisticated  problem  solving  system  that  can  modify  its 
behavior  as  circumstances  change  and  plan  its  own 
&  ]  communication  and  cooperation  strategics  with  other  nodes. 

•  Our  research  has  emphasized  applications  where  there  is  a 
.  natural  spatial  distribution  of  information  and  processing 
requirements  amoog  the  nodes  but  insufficient  local 
g  ,  information  for  any  node  to  make  completely  accurate 
_  processing  and  control  decisions  without  interacting  with 
Si,  ocher  nodes.  An  example  of  this  type  of  application  is  a 
Ll».  distributed  sensor  network  (13,  20.  14],  Our  approach  for 
implementing  these  applications  is  to  have  the  nodes 
cooperate  via  an  iterative,  coroutine  exchange  of  partial  and 
tentative  high-level  results.  In  this  way,  the  system  as  a 
whole  can  function  effectively  even  though  the  nodes 
initially  have  inconsistent  and  ineomplcte  views  of  the 
information  used  in  their  computations  [IS,  16,  17,  1,  2], 


A  key  problem  In  cooperative  distributed  problem 
>g  networks  is  obtaining  sufficient  global  coherence  for 
cooperation  among  the  nodes  (21). 
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coherence  it  not  achieved,  then  the  performance  (speed  and 
accuracy)  of  the  network  can  be  significantly  diminished  as 
a  result  of: 

•  lost  processing  as  nodes  wait  for  something  to  do; 

at  cross-purposes 


wasted  processing  as  nodes  work 
with  one  another; 

redundantly 
efforts; 


applied  processing  as  nodes  duplicate 


e  misallocatioa  of  activities  so  that  important 
portions  of  the  problem  are  either  inaccurately 
solved  or  not  solved  in  timely  fashion. 

These  problems  have  been  observed  in  our  experiments  with 
threo-to-five  node  networks  (15,  17],  We  expect  these 

problems  will  become  even  more  significant  at  we  move  to 
networks  containing  larger  numbers  of  nodes  operating  in 
changing  environments. 

In  this  paper  we  describe  a  decentralized  approach  to 
network  coordination  that  relics  oo  each  node  making 
sophisticated  local  decisions  that  balance  its  own  perceptions 
of  appropriate  problem  solving  activity  with  activities 
deemed  important  by  other  nodes.  Each  node  is  guided  by 
a  high-level  strategic  plan  for  cooperation  among  the  nodes 
in  the  network.  This  strategic  plan,'  which  is  a  form  of 
meta-level  control,  b  represented  as  a  network 

organizational  structure  that  specifies  in  a  general  way  the 
information  and  control  relationships  among  the  nodes. 

In  the  next  section  we  expand  on  the  use  of 

organizational  structuring  as  a  meta-level  network 
coordination  technique.  In  Section  3,  we  briefly  describe 
tbe  local  control  component  of  a  node  and  bow 
organizational  structuring  decisions  influence  this  component. 
Section  4  presents  the  results  of  preliminary  experiments 
with  various  network  problem  solving  strategics  specified  via 
organizational  structuring.  Section  S  discusses  the  prospects 
of  more  complex  forms  of  meta-level  control  usiog 

organizational  structuring.  We  conclude  by  comparing  this 
approach  to  reeent  applications  of  meta-levcl  control  in 
knowledge-based  Artificial  Intelligence  systems. 


□  NETWORK  COORDINATION  VIA 
ORGANIZATIONAL  STRUCTURING 

Network  coordination  is  difficult  in  a  cooperative 
distributed  problem  solving  network  because  limited 
internode  communication  restricts  each  node's  view  ol 
network  problem  solving  aefiviry.  In  tddittoo,  network 
reliability  issues  (which  require  that  the  network's 
performance  degrades  gracefully  if  a  portion  of  the  network 
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(lib)  preclude  the  use  of  a  global  "eoelroUcr"  node.  It  it 
important  that  the  network  coordination  policies  do  not 
consume  more  processing  and  commuaicatioo  resources  than 
the  benefits  derived  from  the  increased  problem  solving 
coherence.  We  believe  that  in  networks  composed  of  even 
a  small  number  of  nodes,  a  complete  analysis  to  determine 
the  detailed  activitcs  at  each  node  is  impractical.  .  Tbe 
computation  and  communication  costs  of  optimally 
determining  the  activities  far  outweigh  tbe  improvement  in 
problem  solving  performance.  Instead,  coordination  in 
distributed  problem  solving  networks  must  sacrifice  tome 
potential  improvement  for  a  less  complex  coordination 
problem. 

Wbat  is  desired  b  n  balance  between  problem  solving 
and  coordination  so  that  tbe  combined  cost  of  both 
activities  is  acceptable.  Tbe  emphasis  is  shifted  from 
optimizing  the  activities  in  tbe  network  to  achieving  an 
acceptable  performance  level  of  the  network  as  a  whole. 
These  policies  must  also  have  enough  flexibility  to  provide 
sufficient  system  robustness  and  reliability  to  respood  to  a 
changing  task  and  hardware  environment,  la  order  for 
network  control  to  satisfy  these  requirements,  it  must  be 
able  to  tolerate  tbe  lack  of  uptodne,  incomplete,  or 
incorrect  control  information  due  to  delays  in  the  receipt  of 
information,  tbe  high  cost  of  acquisition  and  processing  of 
the  information,  and  errors  in  communication  and  processing 
hardware. 

We  feel  that  the  balance  between  local  node  control 
and  network-wide  control  is  a  crucial  aspect  of  the  design 
of  such  decentralized  network  control  policies.  It  is 
unrealistic  to  expect  that  network  control  policies  can  be 
developed  which  are  sufficiently  flexible,  efficient,  and 
require  limited  communication,  while  simultaneously  malting 
all  tbe  control  decisions  for  each  node  in  the  network.  We 
believe  a  node  needs  a  sophisticated  form  of  local  control 
that  permits  it  to  plan  sequences  of  activities  and  to  adapt 
its  plan  based  oo  its  problem  solving  rote  in  tbe  network, 
on  tbe  status  and  role  of  other  nodes  in  tbe  network,  and 
oo  self-awareness  of  its  activities. 

An  organizational  structure  is  used  to  provide  each 
node  with  a  high-level  view  of  problem  solving  in  tbe 
network.  It  specifics  a  general  set  of  oode  responsibilities 
and  node  interaction  patterns  that  is  available  to  all  nodes. 
Included  in  tbe  organizational  structure  are  control  decisions 
that  are  not  quickly  outdated  and  that  pertain  to  a  Urge 
number  of  nodes.  Tbe  sophisticated  local  control 
component  of  each  node  b  responsible  for  eUbo rating  these 
relationships  into  precise  activities  to  be  performed  by  the 
node.  In  this  way  we  have  split  the  network  coordination 
problem  into  two  concurrent  activities  [5]: 

1.  construction  and  maintenance  of  a  network-wide 
organizational  structure; 

2.  continuous  local  elaboration  of  this  structure  into 
precise  activitcs  using  the  local  control  capabilities 
of  each  node. 

Tbe  organizational  structure  provides  a  control  framework 
which  reduces  the  amount  of  control  uncertainty  present  in 
•  node  (due  to  incomplete  or  errorful  local  control 
information)  and  iocr ea sea  the  likelihood  that  the  nodes  will 
be  coherent  in  their  behavior  by  providing  s  general  and 
global  strategy  for  aerwork  problem  solving.  Tbe 
organizational  structuring  approach  to  limiting  control 
uncertainty  Mill  preserves  ■  certain  level  of  control  flciibility 


for  a  node  to  adapt  its  local  control  to  changing  task  and 
environmental  conditions. 

In  order  for  any  network  coordination  policy  to  be 
successful,  it  must  achieve  tbe  following  conditions: 

eaverag*  -  any  given  portion  of  the  overall  problem 

must  be  included  in  the  activities  of  at  least 
ooe  oode; 

ceonectlvity  -  nodes  must  interact  in  a  manner  which 
permits  Use  covering  activities  to  be 
developed  and  integrated  into  an  overall 
solutioo; 

capability  -  coverage  and  connectivity  must  be 

achievable  within  Use  communication  and 
computation  resource  limitations  of  tbe 
network. 

The  organizational  structure  specifics  a  range  of  possible 
coverages  and  connectivity  patterns  that  can  potentially 
satisfy  the  capability  cooditioa.  Using  the  coverage  and 
connectivity  guidelines  specified  in  the  organizational 
structure,  tbe  local  control  component  of  each  node  selects 
a  problem  solving  strategy  based  oo  tbe  dynamics  of  tbe 
specific  local  problem  solving  situation. 

CD  AN  IMPLEMENTATION  OF  ORGANIZATIONAL 
STRUCTURING 

To  provide  a  framework  for  Hudying  tbe  use  of 
organizational  structuring  in  coordinating  the  local  activity 
decisions  of  tbe  nedea  in  n  cooperative  distributed  problem 
solving  network,  we  bsvc  const  rues  ed  the  Distributed 
Vehicle  Monitoring  Testbed  (17).  Tbe  testbed  simulates  a 
network  of  problem  solving  nodes  attempting  to  identify, 
locate,  and  track  patterns  of  vehicles  moving  through  a 
two-dim ensiooal  space  using  signals  detected  by  acoustic 
sensors.  By  varying  parameters  in  the  testbed  that  specify 
the  accuracy  and  range  of  the  scouslic  sensors,  the  acoustic 
signals  that  are  to  be  grouped  together  to  form  patterns  of 
vehicles,  the  power  and  distribution  of  knowledge  amoog 
the  nodes  in  the  network,  and  the  node  and  communication 
topology,  a  wide  variety  of  cooperative  distributed  problem 
solving  situations  can  be  modelled. 

Each  problem  solving  node  is  an 
arcbitecturally-oomplete  Hcarsay-U  system  [7]  (with 
knowledge  sources  appropristc  for  the  task  of  vehicle 

monitoring).  Tbe  basic  Heanayfl  architecture  has  been 
extended  to  include  more  sophisticated  local  control  and  tbe 
capability  of  communicating  hypotheses  and  goals  amoog 
nodes  (},  4).  In  particular,  a  planning  module,  a  goal 

blackboard,  and  communication  knowledge  sources  have 
been  added  (Figure  I).  Goals  arc  created  oo  the  goal 
blackboard  to  indicate  the  node's  infection  to  abstract  and 
extend  hypotheses  on  the  data  blackboard.  The  planner 
can  adapt  the  node's  local  activities  in  respoose  to  the 

potential  processing  activities  of  the  node  (based  on  the 

goals  crested  from  the  node's  hypothesis  structure)  and  to 
externally-directed  requests  from  other  nodes  (communicated 
goals). 

Meta-level  control  via  organizational  structuring  it 
introduced  into  this  node  architecture  through  the  use  of  a 
nonprocedural  sod  dynamically  variable  specification  of  the 
behaviors  of  each  node's  planner,  its  scheduler,  and  its 
communication  knowledge  sources.  These  data  structures. 
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Flgur t  1:  Testbed  Kadc  Architecture. 


called  Interest  uui,  arc  used  to  implement  particular 
network  configurations  sad  coordination  policies.  Each 
interest  area  is  a  list  of  regions  and  levels  of  the  data  or 
goal  blackboard.  Associated  with  each  interest  area  are 
one  or  more  parameters  that  modify  the  behavior  of  the 
node.  There  arc  five  sets  of  interest  areas  for  each  node 
in  the  testbed: 

Lecal  pcnersslng  Interest  areas  influence  the  local  problem 
solving  activities  in  the  node  by  modifying  the  priority 
ratings  of  goals  and  knowledge  source  instantiations  and  the 
behavior  of  the  node's  planner  and  scheduler.  Each  local 
processing  interest  area  has  a  single  parameter  associated 
with  it:  a  weight  specifying  the  importance  of  performing 
local  processing  within  the  interest  area.  By  changing  the 
blackboard  regions  and  their  weights,  problem  solving  can 
be  restricted  to  particular  blackboard  regions  and  levels,  and 
problem  solving  on  particular  regioos  and  levels  can  be 
given  priority  (changing  the  characteristics  of  the  search 
performed  at  a  node).  Knowledge  sources  are  scheduled 
based  on  the  confidence  of  their  input  data,  the  priority  of 
the  type  of  problem  solving  performed  by  the  knowledge 
source,  and  the  rating  of  processing  goals.  The  goal  rating 
is  determined  directly  from  the  interest  area  weight  and 
indirectly  from  the  goal's  relation  to  higher-level  processing 
goals.  Each  node's  local  processing  interest  area 
specification  also  includes  a  subgoaling  specification.  This 
data  Mructurc  Bats  the  blackboard  levels  and  regioos  where 
processing  goals  arc  to  be  subgoaled  and  the  levels,  sites, 
and  ratings  of  the  subgoals.*  Threshold  values  indicating 

Tbt  site  sad  rating  af  a  mbgoal  arc  specified  in  terms  et  the 

•iae  sad  rating  af  the  original  procesring  goal. 


the  minimum  rating  needed  for  a  goal  to  be  subgoaled  are 
also  specified. 

Hypothesis  transmission  Interest  areas  and  gaal  transmission 
Interest  areas  influence  the  behavior  of  the  hypothesis  and 
goal  transmission  knowledge  sources  at  the  node. 
Transmission  interest  areas  are  specified  for  ooe  or  more 
lists  of  nodes  that  are  to  receive  information  (torn  the 
node.  Each  transmission  interest  area  has  a  weight 
specifying  the  importance  of  transmitting  hypotheses  or  goals 
from  that  area  (to  nodes  specified  in  the  node-list)  and  n 
threshold  value  specifying  the  minimum  hypothesis  belief  or 
goal  rating  needed  to  transmit  from  that  area. 

Hypothesis  reception  Interest  areas  and  goal  rcecptlan 
Interest  areas  influence  the  behavior  of  the  hypothesis  and 
goal  reception  knowledge  sources  at  the  node.  Reception 
interest  areas  are  specified  for  lists  of  nodes  that  are  to 
transmit  information  to  the  code.  Each  reception  interest 
area  hat  a  weight  specifying  the  importance  of  receiving  a 
hypothesis  or  goal  in  that  area  (from  a  node  specified  in 
the  node-list),  a  minimum  hypothesis  belief  or  goal  rating 
needed  for  the  hypothesis  or  goal  to  be  accepted,  and  a 
credibility  weight.  The  credibility  weight  parameter  b  used 
to  change  the  belief  of  received  hypotheses  or  the  rating  of 
received  goals.  A  node  can.  reduce  the  effect  of  accepting 
messages  from  a  node  by  lowering  the  belief  or  rating  of 
messages  received  from  that  node.  Each  hypothesis 
reception  interest  area  also  has  a  focusing  weight  parameter 
that  b  used  to  determine  bow  heavily  received  hypotheses 
are  used  in  making  local  problem  solving  focusing  decisions. 
Thu  b  accomplished  by  modifying  the  rating  of  local 
processing  goals  indicating  potential  work  on  these  received 
hypotheses. 


There  are  also  additional  parameters  associated  with 
the  interest  areas  of  each  node  that  specify  the  relative 
weighting  a  node  gives  to  performing  activities  it  perceives 
ms  important  versus  activities  proposed  by  other  nodes.  The 
settings  of  these  parameters  control  the  various  authority 
relationships  among  the  node*  in  the  network. 

These  interest  area  and  authority  specification*  provide 
the  interface  between  the  activity  decisions  made  by  a  node 
and  organizational  structuring  decision*.  They  can  be  used 
to  control  the  amount  of  overlap  and  problem  solving 
redundancy  among  nodes,  the  problem  solving  roles  of 
nodes  (such  is  "integrator",  "specialist",  and  "middle 
manager"),  the  authority  relations  between  nodes,  and  the 
potential  problem  solving  paths  in  the  network.  These  data 
structures  can  be  viewed  as  rudiments  of  a  third  blackboard 
-  an  organizational  blackboard  containing  the  organizational 
roles  and  responsibilities  for  the  node.  A  node's 
organizational  responsibilities  can  be  estabiisbed  and  changed 
by  limply  modifying  these  data  structures.  The  specification 
data  structures  themselves  do  not  provide  an  explicit, 
high-level  representation  of  these  organizational  roles  and 
responsibilities  (this  will  involve  future  work),  but  inucad 
serve  as  s  low-level  "job  description*  of  those  activities  s 
node  should  be  performing  and  those  activities  s  node 
should  be  avoiding. 


IV  TESTBED  EXPERIMENTS  WITH 
ORGANIZATIONAL  STRUCTURING 

In  this  section  we  show  how  different  organizational 
strategics  for  network  problem  solving  can  be  achieved  by 
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appropriate  (ettiap  el  the  interest  area  specifications  ai 
each  aedc  and  bow  ibeae  did cr cat  strategies  perform  in  a 
specific  distributed  problem  solving  situatioo.  Characteristics 
that  were  varied  included: 

e  whether  communication  is  valuator;  (a  node 
transmits  hypotheses  at  its  pleasure),  requested  (a 
node  transmits  hypotheses  only  when  that 

information  a  requested  by  another  node),  or  a 
mixed  Initiative  combination  of  voluntary  and 
requested  hypotheses  (a  node  volunteers  only  its 
highest  rated  hypotheses  and  awaits  requests  before 
transmitting  any  other  hypotheses); 

•  whether  a  node  b  self -directed  or 

externally -directed  in  its  activities  (or  a 
combination  of  both); 

u  whether  hypotheses,  goafs,  or  both  hypotheses  and 
goals  are  used  for  intcraode  coordination. 

the  organizational  strategics  were  evaluated  using  two 
different  network  architectures:  n  laterally-organized, 
four-node  network  with  broadcast  communication  among 
nodes  and  a  hierarchically-organized,  fivo-oode  network  in 
which  the  fifth  node  acts  ns  an  integrating  node.  In  both 
architectures,  the  network  b  structured  so  that  the  nodes 
cooperate  by  exchanging  partial  and  tentative  high-level 
hypotheses. 

The  sensor  configuration  and  input  sensory  signal  data 
used  in  these  experiments  b  shown  in  Figure  2.  Four 
sensors  with  identical  characteristics  and  slightly  overlapping 
ranges  cover  the  monitoring  area.  This  environmental 
scenario  was  designed  to  test  the  network's  ability  to  use 
prediction  to  extend  strongly  sensed  portions  of  an  actual 
vehicle  track  through  weakly  teased  portions  in  the  presence 
of  a  moderately  sensed  “ghost"  track.  Ghost  tracks  are  a 
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particularly  problematic  phenomenon  in  the  vehicle 
monitoring  domain,  causad  by  multiple  propagation  paths  of 
the  actual  signals  and  by  geometrical  ambiguity  in 
combining  signals  from  multiple  vehicles.  The  ghost  track 
in  this  covironmcni  mirrors  the  actual  vehicle  track  for 
eight  consecutive  time  frames.  This  b  unusual.  Typically 
ghosts  behave  as  normal  vehicles  (or  a  brief  period  only  to 
abruptly  disappear  or  to  turn  at  sharp  angles  and  accelerate 
to  infinite  velocity  (JO).  The  gboet  in  ibb  environment 

represents  a  "worst-case*  situation,  appearing  ns  n  normal 
vehicle  with  moderately  strong  sensory  support. 

Synthesis  of  the  answer  map  in  these  experiments 
involves  five  blackboard  levels:  signal  location,  group 
location,  vehicle  location,  vehicle  track,  and  pattern  track. 
The  Signal  location  level  contains  hypotheses  representing 
the  output  of  low-level  aaalysb  of  the  sensory  data.  Each 
signal  location  hypothesis  includes  the  frequency, 
approximate  position,  time  frame,*  and  belief  (based  pertly 
oa  signal  strength  and  sensor  quality)  of  the  delected 

acoustic  signal.  The  group  locotlon  level  contains 

hypotheses  formed  from  harmonically-related  rigaal  location 
hypotheses  at  the  same  time  frame  and  approximately  the 
same  position.  Each  group  location  hypothesis  includes  the 
fundamental  frequency  of  the  related  signals  and  its 
approximate  positron,  time  frame,  and  belief  (a  function  of 
the  beliefs  and  characteristics  of  the  related  signal 
locations).  The  vehicle  location  level  contains  hypotheses 
formed  from  group  location  hypotheses  that  can  be 
combined  to  form  a  particular  type  of  vehicle.  Each 

vehicle  location  hypothesis  includes  the  identity  of  the 
vehicle,  approximate  politico,  tunc  frame,  and  belief.  The 
vehicle  track  level  contains  hypothesized  movements  of 
vehicles  ever  lime.  Each  vehicle  track  hypothesis  includes 
the  identity  of  the  vchide,  its  approximate  porieioa  xt 
successive  time  frames,  end  belief.  The  pattern  track  level 
contains  hypotheses  formed  from  vehicle  track  hypotheses  of 
specific  vehicle  types  that  maintain  a  particular  spatial 
relationship  among  themselves.  Pattern  tracks  were  included 
in  the  testbed  to  investigate  the  effect  of  stroag  constraints 
between  distant  nodes. 

In  the  four-node  network  each  node  is  positioned 
near  one  of  the  sensors  and  receives  signal  location 
hypotheses  from  that  sensor  only.  The  interest  areas  on 
the  organizational  blackboard  of  each  oode  specify  that  it  is 
to  synthesize  its  sensory  data  to  the  vehicle  track  level  and 
transmit  any  of  these  vehicle  track  hypotheses  that  can  be 
extended  into  the  sensory  area  of  another  oode  to  that 
node.  Each  node  b  also  directed  to  attempt  to  generate 
hypotheses  at  both  the  vehicle  track  and  pattern  track  levels 
which  span  the  entire  monitoring  area.  This  means  that 
each  node  b  io  a  race  with  the  other  three  to  generate  the 
complete  answer  map. 

In  the  fivo-oode  network  four  of  the  nodes  are 
positioned  near  ooe  of  the  sen  ion  and  receive  signal 
location  hypotheses  only  from  that  senior.  (Their  signal 
location  input  is  identical  to  the  four-node  network.)  The 
fifth  oode  receives  no  sensory  data.  Instead,  it  is  instructed 
through  interest  areas  oa  the  organizational  blackboard  to 
wotk  only  at  the  vehicle  track  and  pattern  track  levels  with 
vehicle  track  hypotheses  received  from  the  other  four  nodes. 
The  four  nodes  with  sensory  data  are  assigned  the  role  of 
synthesizing  their  signal  location  hypotheses  to  the  vehicle 


Figure  2:  Sensor  Configuration  and  Input  Data.  The  environment  it  sot  tented  cootinuoutly.  Uulcsd.  it  it 

templed  el  ditcrclc  time  intervals  called  time  frame*. 
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uxV  level  Mid  transmitting  them  to  the  fifth  node.  In  the 
five-node  network  configuration,  these  four  nodes  do  not 
work  outside  the  area  of  their  sensory  data  at  any 
blackboard  level  and  do  no  work  at  the  pattern  track  level. 

In  the  four-node  configuration,  voluntary 
tonnunicition  is  obtained  by  providing  each  node  with 
hypothesis  transatWon  interest  areas  specifying  transmission 
of  vehicle  track  hypothesis  so  nodes  with  season  in  the 
area  of  potaible  extension  of  these  hypotheses.  To  keep  the 
node  entirely  self-directed  in  its  local  activity  decisions,  each 
node  b  instructed  not  to  generate  processing  goals  from 
hypotheses  received  from  other  nodes.  The  beliefs  of  the 
received  vehicle  track  hypotheses,  however,  are  not  reduced. 
This  means  that  the  node  can  use  received  information  in 
extending  its  own  hypotheses  without  having  to  find  local 
information  that  can  be  combined  with  the  received 
hypotheses.  This  sepatatioo  of  belief  in  the  data  from 
locating  priority  fits  nicely  into  the  integrated  data-  and 
goal-directed  architecture.  Extern  ally-directed  eootroi  is 
obtained  by  instructing  each  node  to  create  goals  from 
hypotheses  received  from  other  nodes  and  to  use  only  those 
goals  in  its  local  activity  dedsioos.  la  this  strategy,  the 
receipt  of  a  highly-believed  hypothesis  from  another  node 
causes  the  receiving  node  to  tty  its  best  to  find  something 
that  can  be  combined  with  the  received  hypothesis. 
Combined  seif-directed  and  externally-directed  control  is 
obtained  by  instructing  each  node  to  use  goals  generated 
from  both  internal  and  received  hypotheses  in  its  activity 


The  requested  communication  strategy  is  obtained  by 
instructing  each  node  to  process  its  local  sensory  data  to 
the  vehicle  track  level,  but  rather  than  voluntarily 
transmitting  vehicle  back  hypotheses,  any  vehicle  track 
extension  goals  that  are  within  the  sensory  area  of  another 
node  arc  sent  to  that  node.  When  a  node  creates  a 
vehicle  track  hypothesis  that  satisfies  one  of  these  received 
goals  it  transmits  the  hypothesis  to  the  originator  of  the 
goal.  Within  the  requested  communication  strategy, 
self-directed,  externally-directed,  and  combined  cootrol 
strategics  are  obtained  by  instructing  each  node  to  use  goals 
generated  from  internal  hypotheses,  goals  received  from 
other  nodes,  or  both  in  its  local  activity  decisions, 
respectively. 

la  the  live-node  configuration,  mixed-initiative 
communication  is  obtained  by  having  the  worker  nodes 
transmit  only  highly  rated  hypotheses  to  the  integrating 
node.  The  integrating  node  transmits  goals  to  the  worker 
nodes  iofonmag^tbem  of  ils  need  for  additional  dau.  If 
the  received  goals  are  not  used  for  focusing,  the  worker 
nodes  remain  self-directed  in  their  local  activity  decisions, 
only  responding  to  thorn  goals  that  are  achieved  as  1  result 
of  self-directed  processing  activity.  If  the  received  goals  are 
used  for  focusing.  the  worker  nodes  become 
externally-directed  and  attempt  to  achieve  the  received 
goals.  Again,  a  combined  self-  and  externally-directed 
approach  can  also  he  speciHed. 

A.  Remits  ef  Use  feor-eedc  nctwerk  experiments 

Each  of  the  organizational  problem  solving  strategies 
were  tun  on  the  environment  of  Figure  2.  The  network 
was  stopped  when  the  complete  actual  pattern  track 
hypotheses  was  formed  at  one  of  the  four  nodes.  The 
results  arc  shows  in  Table  I.  Whether  the  network  used 
voluntary  or  requested  communication  of  hypotheses  had 


FOUR-NODE  EXPERIMENTS 


Problem 

Solving  Network  Sent  Scat 
Strategy  Cycles  Hyps  Goals 


VH/SD 

33 

23 

0 

VH/ED 

86 

39 

0 

VH/SAED 

79 

45 

0 

RH/SD 

32 

32 

80 

RH/ED 

S3 

35 

133 

RH/SAED 

75 

<0 
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FIVE-NODE  EXPERIMENTS 


Problem 

Solving 

Network 

Sent 

Scot 

Strategy 

Cycles  Hyps 

Goals 

VH/SD 

27 

20 

0 

MH/SD 

25 

18 

14 

MH/ED 

40 

33 

30 

MH/SAED 

29 

22 

18 

Table  1:  Sommery  ef  Network  Experiments. 

Strategies: 

VH  Voluntary  Hypothesis  Communication 

RH  Requested  Hypothesis  Communication 

MH  Mixed-lnilitive  Hypothesis  Communication 
SD  Self-Directed  Cootrol 

ED  Externally-Directed  Control 

SfkED  Combined  Self-  and  Externally-Directed 

Control 

little  effect  on  the  number  of  network  cycles*  required  to 
generate  an  answer.  Whether  the  strategy  was  self-directed 
or  externally-directed  had  a  much  greater  effect  on  network 
performance.  The  completely  externally-directed  strategics 
performed  much  worse  than  the  completely  data-directed 
strategics,  with  the  combined  strategics  in  between. 

Why  does  cxternally-ditected  cootrol  perform  so 
poorly  io  these  experiments?  A  closer  inspection  reveals 
why.  Node  1  (the  node  associated  with  Sensor  1)  senses 
signal  location  hypotheses  in  only  two  time  frames.  Its 
signal  location  hypotheses  are  associated  with  the  false  ghost 
track.  It  does  not  sense  the  actual  vehicle  track  at  all. 
Having  no  other  work  to  perform  Node  1  quickly  forms  a 
two  time-frame  segment  of  the  ghost  track  and  transmits  it 
to  the  other  three  nodes.  This  hypothesis  is  rated  higher 
than  the  stroogly  sensed  signal  location  hypotheses  because 
it  is  at  a  higher  blackboard  level  and  appears  to  be  a 
reasonable  vehicle  track  from  Node  l's  perspective.  Due  to 
their  bias  to  external  direction  the  other  three  nodes 
suspend  work  oo  the  strongly  sensed  lower  level  hypotheses 
of  the  actual  track  and  attempt  to  extend  the  ghost  track, 
resulting  in  inappropriate  knowledge  source  activities  and 
lost  time.  This  is  s  prime  example  of  distraction  (IS). 

To  verify  that  distracting  information  received  from 
Node  1  is  indeed  (he  cause  of  the  poor  performance  of  the 
externally-directed  strategies,  the  requested  communication 
with  both  self-directed  and  externally-directed  control 

• 

A  network  cycle  is  the  cicculioo  of  one  local  proccxring 
knowledge  source  si  (Kb  node  in  tbc  network.  If  a  node  hu 
oo  work  to  perform  during  a  cycle,  its  potential  knowledge 
senircc  execution  is  lest. 


experiment  was  tenia  with  Node  I  disabled.  Hie  number 
el  network  cycles  was  reduced  from  75  with  Node  1  to  38 
without  Node  1.  The  network  actually  performs  much 
better  without  Node  I.  even  though  the  remaining  nodes 
still  receive  an  signal  location  hypotheses  associated  with  the 
ghost  track. 

B.  Results  of  the  ftve-onde  network  experiments 

The  results  of  the  five-node  experiments  are  also 
shown  in  Table  1.  In  this  case  the  network  was  stopped 
when  the  complete  actual  partem  track  hypothesis  was 
formed  at  the  integrating  node  Whether  the  network  used 
voluntary  or  mixed-initiative  communication  of  hypotheses 
again  bad  little  effect  on  the  number  of  network  cycles 
required  to  generate  an  answer.  As  with  the  four-node 
network  experiments,  whether  the  strategy  was  self-directed 
or  cateraally-directcd  bad  a  much  greater  effect  on  network 
performance.  The  completely  externally-directed  strategics 
performed  much  worse  than  the  completely  dau-directcd 
strategics,  with  the  combined  itrategics  in  between. 

In  this  case  the  information  received  by  the 
integrating  node  (Node  5)  from  Node  1  causes  it  to  make 
inappropriate  coordination  decisions  for  the  other  three 
worker  nodes.  In  place  of  distracting  hypotheses  received 
directly  from  Node  1.  distraction  of  the  worker  nodes  takes 
the  indirect  form  of  distracting  goals  received  from  Node  5. 

The  mixed-initiative  communication  with 

externally-directed  control  experiment  was  renin  with  Node 
1  disabled.  Again  the  loss  of  Node  1  improved  the 
performance  of  the  network  by  eliminating  its  distracting 
influence.  The  number  of  network  cycles  was  reduced  from 
40  with  Node  1  to  29  without  Node  1.  The  network  again 
performed  much  better  without  the  distractions  from  Node 
I. 

C.  Comparing  the  four -coda  and  flvc-oodt  experiments 

When  the  additional  processing  provided  by  the  fifth 
node  is  taken  into  account,  the  performance  of  the  lateral 
four-node  network  was  basically  identical  with  the 
performance  of  the  hierarchical  five-node  network  in 
comparable  sell-directed  experiments  (Table  2).  The 
five-node  network  does  appear  to  perform  better  than  the 
four-node  network  in  the  externally-directed  strategies. 
When  a  node  in  the  four-node  network  receives  distracting 

Problem  Normalised 

Solving  Four-Node  Five-Node 

Strategy  Network  Cycles  Network  Cycles 


Vtl/SD 

26.4 

27 

R-MH/SD 

25.6 

25 

R-MH/ED 

66.4 

40 

R-MH/S&ED 

60.0 

29 

Table  2:  Network  Cycle  Comparison  of  the  Four-  and 
Flve-Nedc  Experiments. 


Strategies: 


Voluntary  Hypothesis  Communication 
Requested  or  Mixed-Initiative  Hypothesis 
Communication 
Sell-Directed  Control 
Externally-Directed  Control 
Combined  Self-  and  Externally-Directed  Control 
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information  it  generally  processes  it  to  the  pattern  track 
level  before  resuming  work  on  its  own  lower  level 
hypotheses  (due  to  the  generally  higher  belief  associated 
with  higher  abstraction  levels).  A  worker  node  in  the 
five-node  network  only  processes  distracting  information  to 
the  vehicle  track  level,  and  then  tends  the  information  on 
to  the  integrating  node.  Thus  the  worker  node  can  resume 
its  activities  sooner  than  a  node  in  the  four-node 
architecture.  The  integrating  node,  while  distracted,  it  oot 
synthesizing  low  level  data  and  is  therefore  less  affected  by 
the  distracting  information.  By  dividing  the  additional  work 
caused  by  the  distracting  hypotheses  between  nodes  with 
different  problem  solving  responsibilities,  the  overall  effect 
of  distraction  is  reduced. 

While  the  experiments  reported  in  this  section  indicate 
that  different  network  problem  solving  strategies  specified 
via  organizational  structures  have  different  problem  solving 
characteristics,  they  do  oot  provide  sufficient  data  for 
drawing  any  conclusions  on  the  particular  benefits  of 
particular  organizational  strategies.  These  experiments  were 
performed  with  a  tingle  environmental  scenario  with  fairly 
unrestricted  communication.  Different  problem  solving 
characteristics  may  favor  different  organizational  strategics. 
Particularly  important  is  exploration  of  larger  networks. 
(These  experiments  are  just  beginning.)  A  four  or  five 
node  network  simply  bas  too  few  nodes  for  organizational 
structuring  decisions  to  have  a  significant  impact. 
Experiments  with  tens  or  even  hundreds  of  nodes  are 
needed  before  the  full  effect  of  organizational  structuring 
will  be  seen. 


V  MORE  COMPLEX  META-LEVEL  CONTROL 

While  organizational  structuring  could  be  performed 
by  directly  changing  the  interest  areas  of  each  node  (the 
approach  used  in  the  experiments  reported  here),  an  indirect 
approach  allows  the  node  to  adopt  or  reject  its 

organizational  roles. 

Instead  of  modifying  the  specifications  directly,  a 
second,  separate  set  of  node  activity  specification  data 
structures  is  kept  at  each  node.  The  origioal  interest  areas 
remain  as  the  behavioral  command  ccotcr  of  the  node. 
Their  lettings  directly  influence  the  node's  activities.  Tbc 
second  specifications  set  forms  the  lowest  level  of  the 
full-fledged  organizational  blackboard.  They  are  the  result 
of  elaborating  bigher-levet  organizational  roles  and 
responsibilities  into  ao  'organizational  job  description".  The 
complete  structure  of  this  organizational  blackboard,  and  the 
processing  needed  to  perform  tbc  elaboration,  remain  an 

open  research  issue.  Wbat  is  important  here  is  that  the 
specifications  directly  controlling  the  behavior  of  a  node  and 
the  behavior  suggested  by  the  organizational  stnicture  arc 
separated.  The  node  undertakes  its  organizational  activities 
only  by  transferring  organizational  specifications  into  its 
interest  areas. 

The  activities  of  a  node  should  also  be  influenced  by 
its  potential  for  performing  them.  A  node  is  continually 
receiving  sensory  data  and  hypotheses  from  other  nodes. 
This  information  provides  numerous  opportunities  for  local 
node  activities.  However,  the  node's  interest  areas  (possibly 
set  from  the  organizational  blackboard)  may  be  strongly 
opposed  to  performing  these  activities.  The  node's  potential 
for  work  is  represented  oo  a  fourth  blackboard,  the  local 
node  focusing  blackboard.  This  blackboard  specifics  where 
the  node  perceives  there  is  substantial  work  it  it  able  to 
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perform.  As  with  Ike  organizational  specifications,  these 
focusing  specifications  can  be  transferred  to  the  node's 
interest  areas,  at  which  point  the  node  win  actively  pursue 
these  activities. 

When  the  roles  and  responsibilities  represented  in  the 
organizational  blackboard  are  in  conflict  with  the  criteria  oh 
the  local  node  focusing  blackboard,  an  arbiter  for 
determining  the  actual  interest  areas  is  needed  (Figure  3). 
Favoring  the  specifications  on  the  organizational  blackboard 
make  the  node's  behavior  more  in  line  with  the 
organizational  structuring  decisions  (more  of  a  ‘company 
node9),  while  favoring  the  local  node  focusing  specifications 
make  the  node  more  responsive  to  its  ability  to  immediately 
perform  work  on  quality  data.  Such  nedc  skepticism  is  an 
important  source  of  network  robustness  when  organizational 
structuring  derisions  arc  made  using  incomplete  and 
inaccurate  information.  A  skeptical  node's  local  activity 
decisions  are  constantly  pulled  in  two  directions:  toward  the 
responsibilities  specified  by  the  organizational  structure  and 
toward  the  activities  suggested  by  its  local  data  and 
interactions  with  other  nodes.  Tbe  tension  between  these 
two  directions  can  lead  to  an  increase  in  the  network's 
ability  to  tolerate  organizational  control  errors.  If  a  node's 
organizational  responsibilities  are  inappropriate  to  its 
potential  activities,  tbe  node  can  proceed  with  locally 
generated  activities.  Similarly,  organizational  responsibilities 
can  be  ignored  by  nodes  which  possess  strong  information 
to  tbe  contrary;  a  node  with  a  unique  perspective  is  not 
necessarily  stifled  by  an  uninformed  majority.  The  degree 
of  node  skepticism  exhibited  by  a  node  should  dynamically 
change  according  tbe  node's  perception  of  the 
appropriateness  of  tbe  organizational  structure.  If  a  node 
has  no  reason  to  doubt  the  organization  structure  it  should 
be  receptive  to  organizatiooally-sperified  activities.  As  a 


Flgurt  3:  Tht  Organizational  and  Local  Nodr  Focusing 
Blackboards  and  Node  Skepticism. 


node  becomes  skeptical  of  the  organizational  structure,  it 
should  switch  to  its  own  local  activities,  and  disregard 
organizational  activities  which  arc  in  conflict  with  its  local 
activities  (Sj. 

The  existence  of  the  organizational  and  local  node 
focusing  blackboards  also  help  indicate  when  tbe  portion  of 
the  network  organizational  structure  relating  to  tbe  node 
needs  changing.  A  strong  mismatch  between  tbe  two 
blackboards  is  a  sign  of  trouble,  and  the  information 
contained  in  the  focusing  blackboard  can  be  a  valuable  aid 
in  determining  new  roles  and  responsibilities. 

Three  additional  components  are  relevant  to  tbe 
organizational  structuring  approach  to  network  coordination: 

1.  A  distributed  task  aOacatlm  component  for 

deciding  what  dynamic  information  and 
processing  goals  should  be  transmitted  among  tbe 
nodes.  Given  tbe  high-level  strategic  plan  for 

the  allocation  of  activities  and  control 
responsibilities  among  nodes  (tbe  organizational 
structure)  there  b  still  a  need  to  make  more 
localized,  tactieal  derisions  that  balance  tbe 
activities  among  the  nodes  based  on  tbe  dynamics 
of  tbe  current  problem  solving  situation  [19J. 

2.  A  knowledge-based  fault-diagnosis  component  for 
detecting  and  locating  inappropriate  system 
behavior.  Wc  are  looking  to  not  only  isolate 
problems  caused  by  hardware  errors,  but  also 
inappropriate  settings  of  tbe  problem  solving 
parameters  that  specify  strategic  and  tactical 
network  coordination  [12]. 

3.  An  organizational  self-design  component  for 
initially  developing  an  organizational  structure 
and  for  modifying  that  structure  to  reduce  tbe 
effect  of  hardware  errors  or  an  inappropriate 
organizational  structure  (both  recognized  by  tbe 
fault-diagnosis  component).  When  a  hardware 
error  is  detected,  tbe  the  network  coordination 
policy  needs  to  be  modified  to  that  tbe  offending 
hardware  and  resulting  incorrect  processing  docs 
not  distract  problem  solving  in  other  pans  of  tbe 
network  and  to  establish  alternative  paths  for 
generating  a  more  accurate  version  of  the  needed 
information  wherever  possible.  When  the 
organizational  structure  becomes  inappropriate 
(due  to  changes  in  tbe  internal  or  external 
environment  of  the  distributed  problem  solving 
network)  plausible  alternative  structures  need  to 
be  determined  and  evaluated  as  potential 
candidates  for  network  reorganization  (S). 

VI  CONCLUSION 

Distributed  problem  solving  networks  provide  an 
interesting  application  area  for  meta-level  control  through 
the  use  of  organizational  structuring.  Tbe  organizational 
structure  provides  each  node  with  a  high-level  view  of 
problem  solving  in  tbe  network.  The  sophisticated  local 
control  component  of  each  node  is  responsible  (or 
elaborating  these  relationships  into  precise  activities  to  be 
performed  by  tbe  node,  based  on  the  node's  problem 
solving  rote  in  the  network,  on  the  status  and 
organizational  roles  of  other  nodes  in  the  network,  and  on 
sell-awareness  ol  the  node's  activities.  The  balance  between 
local  node  control  and  organizational  control  is  a  crucial 
aspect  ol  this  approach. 
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W«  have  implemented  this  approach  ia  (be  Distributed 
Vehicle  Monitoring  Testbed  (17J.  Our  preliminary 
experiments  using  the  testbed  indicate  that  by  adjusting  the 
organizational  structure  different  network  problem  solving 
strategies  can  be  obtained.  The  aeat  step  in  this  research 
is  experiments tioa  with  larges  distributed  problem  solving 
networks  where  the  effects  of  organizational  structuring 
decisions  will  become  Sncrcaangly  significant. 

It  is  interesting  to  note  that  the  themes  of  this 
research.  which  advocate  the  interplay  between 
organizational  control  and  sophisticated  local  node  control, 
arc  dose  in  emphasis  to  recent  trends  emphasizing 
meu-levcl  control  and  sophisticated  planning  in 
knowledge-based  Artificial  Intelligence  systems  {II,  6,  9,  22, 
8].  The  introduction  of  aa  organizational-level  of  control 
into  distributed  problem  solving  b  an  example  of  the  use  of 
meta-lcvd  control  to  coordinate  activity  in  a  complex 
system.  As  Nilsaon  has  noted,  the  field  of  distributed 
Artificial  Intelligence  serves  to  illuminate  basic  Artificial 
Intelligence  issues  pgj.  In  thb  case,  the  need  to  cootrol 
the  uncertainty  inherent  with  semi-autonomous  problem 
solving  agents  possessing  only  a  local  and  possibly  crrorful 
view  of  'the  global  state  of  problem  wiving  b  very  similiar 
so  the  cootrol  problems  that  are  being  faced  in  the 

development  of  the  new  generation  of  knowlcdgo- based 
problem  solving  systems  which  have  significantly  larger  and 
awrc  diverse  knowledge  bases. 

Our  use  of  meta-lcvd  cootrol  with  its  emphasis  on 
providing  general  guidelines  for  acceptable  problem  solving 
behavior  differs  from  the  work  of  Hayes-Roth,  Davis,  and 
Stcfik  which  uses  meu-levcl  cootrol  to  make  specific 

strategic  problem  solving  decisions  (11,  6,  22].  In  our 

approach  the  specific  strategy  decisions  are  made  by  the 

local  control  component  of  a  node  using  the  guidelines 
provided  by  the  mcta-levd  organizational  structure. 
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